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Welcome  to  our  Ninth  Annual  Acquisition  Research  Symposium!  This  event  is  the 
highlight  of  the  year  for  the  Acquisition  Research  Program  (ARP)  here  at  the  Naval 
Postgraduate  School  (NPS)  because  it  showcases  the  findings  of  recently  completed 
research  projects — and  that  research  activity  has  been  prolific!  Since  the  ARP’s  founding  in 
2003,  over  800  original  research  reports  have  been  added  to  the  acquisition  body  of 
knowledge.  We  continue  to  add  to  that  library,  located  online  at 

www.acquisitionresearch.net,  at  a  rate  of  roughly  140  reports  per  year.  This  activity  has 
engaged  researchers  at  over  60  universities  and  other  institutions,  greatly  enhancing  the 
diversity  of  thought  brought  to  bear  on  the  business  activities  of  the  DoD. 

We  generate  this  level  of  activity  in  three  ways.  First,  we  solicit  research  topics  from 
academia  and  other  institutions  through  an  annual  Broad  Agency  Announcement, 
sponsored  by  the  USD(AT&L).  Second,  we  issue  an  annual  internal  call  for  proposals  to 
seek  NPS  faculty  research  supporting  the  interests  of  our  program  sponsors.  Finally,  we 
serve  as  a  “broker”  to  market  specific  research  topics  identified  by  our  sponsors  to  NPS 
graduate  students.  This  three-pronged  approach  provides  for  a  rich  and  broad  diversity  of 
scholarly  rigor  mixed  with  a  good  blend  of  practitioner  experience  in  the  field  of  acquisition. 
We  are  grateful  to  those  of  you  who  have  contributed  to  our  research  program  in  the  past 
and  hope  this  symposium  will  spark  even  more  participation. 

We  encourage  you  to  be  active  participants  at  the  symposium.  Indeed,  active 
participation  has  been  the  hallmark  of  previous  symposia.  We  purposely  limit  attendance  to 
350  people  to  encourage  just  that.  In  addition,  this  forum  is  unique  in  its  effort  to  bring 
scholars  and  practitioners  together  around  acquisition  research  that  is  both  relevant  in 
application  and  rigorous  in  method.  Seldom  will  you  get  the  opportunity  to  interact  with  so 
many  top  DoD  acquisition  officials  and  acquisition  researchers.  We  encourage  dialogue  both 
in  the  formal  panel  sessions  and  in  the  many  opportunities  we  make  available  at  meals, 
breaks,  and  the  day-ending  socials.  Many  of  our  researchers  use  these  occasions  to 
establish  new  teaming  arrangements  for  future  research  work.  In  the  words  of  one  senior 
government  official,  “I  would  not  miss  this  symposium  for  the  world  as  it  is  the  best  forum 
I’ve  found  for  catching  up  on  acquisition  issues  and  learning  from  the  great  presenters.” 

We  expect  affordability  to  be  a  major  focus  at  this  year’s  event.  It  is  a  central  tenet  of 
the  DoD’s  Better  Buying  Power  initiatives,  and  budget  projections  indicate  it  will  continue  to 
be  important  as  the  nation  works  its  way  out  of  the  recession.  This  suggests  that  research 
with  a  focus  on  affordability  will  be  of  great  interest  to  the  DoD  leadership  in  the  year  to 
come.  Whether  you’re  a  practitioner  or  scholar,  we  invite  you  to  participate  in  that  research. 
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Mary  Lacey — Lacey  is  the  deputy  assistant  secretary  of  the  Navy  for  Research,  Development,  Test, 
And  Evaluation  (RDT&E).  She  is  the  senior  civilian  and  serves  as  the  senior  advisor  to  the 
ASN(RD&A)  for  research,  development,  test,  and  evaluation,  and  system  engineering.  She  has 
oversight  responsibility  for  all  science  and  engineering,  test  and  evaluation,  modeling  and  simulation, 
chief  systems  engineering  policy,  practices,  and  processes  for  ASN(RD&A). 

Lacey  also  oversees  the  Department  of  the  Navy  (DoN)  chief  systems  engineering  position  and 
the  DoN  deputy  for  test  and  evaluation.  She  is  the  functional  acquisition  workforce  competency  leader 
for  systems  engineering,  and  she  is  responsible  for  the  long-term  stewardship  of  Naval  Laboratories 
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Lacey  earned  a  Bachelor  of  Science  degree  in  mechanical  engineering  from  the  University  of 
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awards  include  the  Presidential  Rank  Distinguished  and  Meritorious  Executive,  DoD  Distinguished 
Civilian  Service,  the  Navy  Distinguished  Public  Service,  Superior  Civilian  Service,  Women  in  Science 
and  Engineering  Lifetime  Achievement,  University  of  Maryland  Distinguished  Engineering  Alumna, 
and  the  Federal  Laboratory  Consortium  Laboratory  Director  of  the  Year.  Lacey  serves  on  the 
University  of  Maryland  School  of  Engineering  Board  of  Visitors,  the  Women  in  Engineering  Advisory 
Board,  and  the  International  Council  of  Systems  Engineering  Foundation  Board. 

Lacey  brings  to  her  position  a  wealth  of  experience  and  valuable  insight  into  civilian  workforce 
issues.  Throughout  her  career,  Lacey  has  been  actively  involved  in  engineering  workforce 
development.  She  continues  to  serve  as  a  mentor  and  advisor  to  engineering  professionals.  She  has 
proven  expertise  in  managing  large,  diverse  workforces  and  in  leading  and  sustaining 
transformational  change. 
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Postgraduate  School  (NPS)  in  Monterey,  CA.  He  teaches  and  conducts  research  in  the  design  of 
enterprise  systems,  systems  modeling,  and  system  architecture.  He  has  published  over  50  technical 
articles  on  these  topics,  including  a  textbook  entitled  Design  of  Enterprise  Systems:  Theory,  Methods, 
and  Architecture.  Prior  to  joining  NPS,  he  was  at  Florida  International  University  in  Miami,  FL. 
[regiache@nps.edu] 

Abstract 

The  Department  of  Defense  (DoD)  has  an  enterprise  architectural  vision  and  an 
accompanying  transformation  plan.  The  enterprise  transformation  plan  describes  multiple 
individual  projects  and  systems  that  collectively  deliver  the  desired  capabilities  and  enterprise 
architecture.  These  projects  are  performed  over  planning  horizons  that  span  several  years  or 
more.  Deciding  on  what  projects  to  invest  in,  when  to  invest  in  them,  and  whether  to  continue 
the  investment  as  time  progresses  is  a  difficult  problem  because  of  the  uncertainty  involved  in 
the  operational  environment,  the  technology,  and  the  associated  project  risks.  This  paper 
argues  that  enterprise  systems  acquisition  can  be  modeled  using  real  options  to  obtain 
project  valuations  that  consider  the  environmental  uncertainty  and  guide  acquisition 
decisions.  Moreover,  because  the  enterprise  architecture  involves  many  projects  that  are 
interdependent,  a  portfolio  investment  approach  is  called  for.  We  present  a  real  options 
framework  to  plan  a  portfolio  of  projects  as  a  collection  of  compound  real  options.  We 
illustrate  how  the  model  can  be  applied  in  a  case  study  derived  from  the  DoD’s 
transformation  plan.  The  model  and  method  contribute  an  approach  to  value  a  portfolio  of 
projects  that  intentionally  creates  options  to  preserve  decision  flexibility  and  acquire  the  target 
architecture’s  capabilities  at  lower  cost  and  risk. 

Introduction 

The  Department  of  Defense  (DoD)  is  a  very  large  and  complex  organization  that 
undergoes  nearly  constant  transformation  in  the  “small”  due  to  continuous  improvement 
efforts  as  well  as  larger,  more  transformative  changes  due  to  large-scale  projects.  These 
projects  are  performed  to  improve  operational  effectiveness  and  efficiency,  as  well  as  to 
acquire  new  capabilities.  Managing  transformation  on  such  a  large  scale  is  a  formidable 
challenge.  As  is  common  in  many  large  organizations,  the  DoD  has  developed  a  hierarchical 
planning  process  that  aligns  projects  to  strategic  goals.  The  DoD  has  three  documents,  or 
plans,  that  are  used  to  guide  transformation.1  The  three  documents  relevant  to  transforming 
the  business  systems  and  processes  in  the  DoD  are  the  Strategic  Management  Plan  (SMP), 
Business  Enterprise  Architecture  (BEA),  and  Enterprise  Transformation  Plan  (ETP).  The 
SMP  is  the  highest  level  plan  for  improving  the  DoD’s  business  operations.  It  is  a  living 
document  that  is  updated  each  year  to  reflect  guidance  from  the  Quadrennial  Defense 
Review  (QDR),  external  influences,  and  internal  changes  based  on  lessons  learned.  It  aligns 
the  DoD’s  business  goals  with  the  DoD’s  overall  strategic  goals.  The  BEA  is  a  high-level 
design  specifying  the  DoD’s  business  environment  using  the  models  of  the  Department  of 
Defense  Architecture  Framework  (DoDAF).  It  is  mandated  that  all  DoD  systems  use  the 
DoDAF  to  describe  the  system  in  a  common  format  that  will  promote  consistency  and 
interoperability  across  the  DoD.  The  BEA  uses  five  core  business  missions  to  set  priorities 
and  to  align  business  transformation.  The  five  core  business  missions  are  financial 
management  (FM),  human  resource  management  (HRM),  material  supply  and  service 


1  These  documents  were  originally  developed  by  the  Business  Transformation  Agency  (BTA),  but  this  agency 
was  disestablished  on  September  30,  201 1 ,  and  its  mission  and  function  were  assumed  by  the  Office  of  the 
Deputy  Chief  Management  Officer  (DCMO). 
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management  (MSSM),  real  property  and  installations  lifecycle  management  (RPILM),  and 
weapon  systems  lifecycle  management  (WSLM).  The  ETP  describes  the  acquisition  strategy 
for  new  systems  that  make  up  the  target  enterprise  architecture. 

The  DoD  BEA  is  acknowledged  to  be  a  work-in-progress  in  that  the  architectural 
vision  is  updated  on  a  yearly  basis,  currently  at  Version  8.0.  A  changing  target  architecture 
is  not  indicative  of  a  planning  failure  but  recognition  that  the  DoD  is  a  complex,  socio- 
technical,  and  open  system  that  operates  in  a  dynamic  and  uncertain  environment 
(Giachetti,  2010).  As  an  open  system,  it  must  constantly  change  to  adapt  to  its  changing 
environment,  whether  the  changes  are  in  budget,  manpower,  or  the  global  threat 
environment. 

Achievement  of  strategic  goals  and  acquisition  of  systems  to  realize  those  goals 
under  uncertainty  involves  two  strategies.  One  strategy  is  to  minimize  risk  by  testing  and 
validating  architectures  prior  to  deployment.  However,  even  with  greater  testing  of 
architectures,  because  testing  helps  resolve  some  of  the  internal  uncertainty,  significant 
external  uncertainty  will  remain.  The  second  strategy  is  to  continuously  adapt  the 
architecture  deployment  plans  to  react  to  changes  in  the  environment  as  well  as  the 
resolution  of  uncertainty.  Here  we  address  this  last  strategy  with  real  options  theory. 

Real  options  are  both  a  means  to  value  investments  as  well  as  a  means  to  define 
flexibility  in  system  deployment  (Trigeorgis,  2001).  Koenig  (2009)  discussed  the  high  level  of 
uncertainty  and,  hence,  risk  associated  with  conventional  engineering  economic  analysis  of 
projects  that  have  long  operational  lives.  He  suggested  that  the  DoD  environment  is  actually 
rich  with  options,  but  until  now,  there  has  been  no  quantitative  means  to  value  them  and 
incorporate  them  into  the  acquisition  decision  process.  In  fact,  quite  an  extensive  amount  of 
research  has  been  conducted  on  the  proposed  or  actual  use  of  real  options  with  project 
planning  and  acquisition. 

Wang  and  de  Neufville  (2006)  distinguished  between  real  options  on  a  project  and 
real  options  in  a  project.  Real  options  on  a  project  are  the  standard  options  available  to 
almost  all  projects,  including  delay,  abandon,  expand,  and  contract.  Real  options  in  a  project 
are  those  options  designed  into  the  system  architecture.  Most  research  addresses  real 
options  on  projects  with  some  exceptions,  such  as  Engel  and  Browning  (2006),  who 
investigated  how  to  design  adaptable  architectures.  In  the  military  environment,  Uchytil, 
Housel,  Horn,  Mun,  and  Tarantino  (2007)  combined  real  options  with  knowledge  value 
analysis  (KVA)  to  analyze  four  different  options  for  the  AEGIS  system. 

Interest  has  expanded  beyond  valuing  individual  projects  by  the  real  options  method 
to  the  valuation  of  a  portfolio  of  options.  Bardhan,  Bagchi,  and  Soustad  (2004)  developed  an 
approach  to  prioritize  information  technology  projects  using  real  options.  This  approach  and 
several  similar  approaches  value  the  projects  individual  in  the  portfolio,  which  may  miss  the 
important  interactions  between  projects.  Brosch  (2008)  investigated  a  mathematical  model 
to  simultaneously  value  a  portfolio  of  real  options.  His  approach  provided  some  insight,  but 
the  mathematical  formulation  was  complex,  and  he  could  only  solve  the  model  for  trivial 
problems. 

This  paper  contributes  to  the  literature  on  real  options  valuation  in  acquisition  by 
presenting  a  model  to  value  a  portfolio  of  options  rather  than  individual  options  inside  the 
portfolio.  This  is  accomplished  by  adopting  a  switching  formulation  of  real  options  in  a 
discrete  stochastic  mathematical  program.  The  model  is  solved  with  a  Monte  Carlo 
simulation-based  algorithm. 
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The  paper  is  organized  as  follows:  The  background  discusses  transformation  and 
enterprise  architecture  as  conducted  in  the  DoD.  The  real  option  portfolio  model  and  method 
are  presented  next.  An  illustrative  example  is  solved  using  the  model  and  method  to  derive 
a  portfolio.  The  paper  concludes  by  highlighting  the  main  contributions  and  discussing  future 
research. 


Decision-Making  Method  and  Model 

Figure  1  shows  the  flowchart  for  creating  real  options  on  the  enterprise  projects  and 
then  selecting  a  portfolio  of  those  projects  to  maximize  expected  value  to  the  organization. 
The  model  supports  an  acquisition  decision-maker  in  that  it  does  not  automate  the  decision 
but  provides  a  valuation  of  the  portfolio.  In  this  section,  we  describe  the  method,  and  in  the 
next  section,  we  illustrate  the  method  with  an  example. 


Figure  1.  Portfolio  Construction  Algorithm 
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Convert  Projects  to  Real  Options 

The  model  assumes  that  there  are  projects  already  identified,  but  in  a  traditional  way, 
without  consideration  of  decision-making  flexibility.  The  first  step  is  to  convert  the  projects  to 
real  options  by  examining  the  project  and  mapping  an  option  type  onto  the  project.  Option 
types  common  to  almost  all  projects  are  delay,  abandon,  expand,  and  contract  in  addition  to 
doing  the  project  as  planned  (Trigeorgis,  1996). 

Interdependence 

When  conducting  multiple  projects  simultaneously  in  an  enterprise,  we  need  to  take 
into  consideration  any  interdependence  between  the  projects.  Project  interdependencies 
arise  due  to  the  use  of  common  resources,  benefits  derived  from  the  projects,  or  technical 
considerations  (Dickinson,  Thornton,  &  Graves,  2001).  The  interdependencies  can  result  in 
either  mutually  exclusive  projects,  contingent  projects,  or  a  correlation  between  project 
success  or  failure.  Mutually  exclusive  projects  are  either-or  projects.  For  example,  in 
enterprise  transformation,  there  may  be  two  different  projects  to  achieve  a  single  goal,  and 
only  one  of  those  projects  will  be  performed.  In  our  model,  we  enforced  mutually  exclusive 
projects  through  constraints.  Contingent  projects  are  when  one  project  can  only  be  done  if 
another  project  is  done.  This  is  common  with  infrastructure-type  projects  that  establish 
growth  opportunities  that  are  provided  by  subsequent  projects.  Such  contingent  projects  are 
modeled  as  compound  real  options  on  the  infrastructure  project  because  it  provides  an 
opportunity  to  realize  the  future  growth  options.  Two  projects  might  be  highly  correlated 
such  that  if  the  value  of  one  project  increases  (decreases),  then  the  value  of  the  second 
project  increases  (decreases).  Positive  effects  are  when  two  projects  are  complementary 
and  a  synergistic  effect  increases  the  value  beyond  what  either  project  would  provide 
singularly.  A  negative  effect  is  when  there  is  cannibalization  or  the  projects  overlap  such  that 
by  doing  so,  both  the  total  value  is  less  than  if  they  were  added  together.  We  model  these 
interdependencies  as  a  correlation  pxy  between  the  cash  flows  or  benefits  of  project  x  and 
project  y.  The  correlation  may  be  positive  or  negative  such  that  pxy  e  [-1 ,1], 

Portfolio  Optimization  Model 

The  portfolio  model  is  a  stochastic  mathematical  model  that  finds  the  optimal 
sequence  of  project  investment  decisions  to  maximize  the  total  net  present  value  of  the 
project  portfolio  over  the  planning  horizon.  We  utilized  a  switching  formulation  of  the  options 
adapted  from  the  work  of  Kulaitilaka  (1998)  that  was  later  extended  by  Brosch  (2008).  In  the 
switching  formulation,  each  project  can  operate  in  one  of  several  operating  modes. 

Switching  from  one  operating  mode  to  another  is  the  exercise  of  an  option.  Let  a,  denote  the 
option  to  operate  the  project  in  mode  /.  We  let  a?  denote  the  default  operation  mode  of  delay 
or  equivalently  postponing  investment.  Initially,  every  project  is  in  mode  a?,  meaning  that  it  is 
not  being  conducted.  The  other  modes  can  be  defined  to  represent  different  options.  For 
example,  we  could  define  a2  to  denote  the  operation  mode  of  a  pilot  project  and  a3  to  denote 
investment  in  a  full-scale  deployment.  Given  these  definitions,  different  switches  denote 
different  options.  With  the  three  operating  modes,  the  options  available  are  shown  in  Table 
1. 
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Table  1.  Switching  0 

ptions 

Current  Operating 
Mode 

Next  Operating 
Mode 

Option 

a?  -  delay 

a?  -  delay 

Delay 

a?  -  delay 

a2-  pilot 

Pilot 

ai  -  delay 

a3  -  full  scale 

Full  scale  project 

a2-  pilot 

a?  -  delay 

Abandon 

a2  -  pilot 

aa  —  full  scale 

Expand 

a3  -  full  scale 

ai  -  delay 

Abandon 

a3  -  full  scale 

a2  -  pilot 

contract 

The  decision  is  which  operating  mode  switches  to  make  in  each  time  period.  Let 
xp,t,a,d  denote  the  binary  decision  variable  for  project  p  in  time  period  t  of  whether  to  switch 
from  operating  mode  a  to  operating  model  a.  The  decision-maker  seeks  to  maximize  the 
value  of  the  portfolio  so  the  value  of  all  the  options  in  the  portfolio  must  be  calculated. 

The  value  of  an  option  depends  on  the  investment  in  the  option  and  the  expected 
cash  flow  generated  from  the  option.  The  investment  cost  in  the  project  option  is  denoted 
Ip,t,a,& ■  One  repercussion  of  creating  options  on  projects  is  the  need  to  determine  the 
investment  cost  of  all  possible  switches,  as  given  in  Table  1 .  For  example,  the  model  allows 
for  a  cost  or  investment  to  abandon  a  project  option.  This  would  entail  costs  associated  with 
dismantling  technologies,  reverting  to  a  previous  operating  mode,  laying  off  project 
employees,  and  storing  project  materials. 

The  cash  flow  generated  from  a  project  p  in  time  period  t  and  operating  mode  a  is 

Cp,t,a  —  fp,a  T  Ap,t@p,t'  0  ) 


where  fPA  is  the  fixed  return  and  Apa  is  the  variable  return  rate  that  depends  on  the 
underlying  asset,  which  is  denoted  by  9pt.  The  random  variable  9pt  represents  the 
uncertain  future  value  of  the  underlying  asset,  which  is  the  project  p  during  time  period  t. 


The  value  is  calculated  backwards  from  the  end  of  the  planning  horizon  to  the  first 
time  period.  Value  for  the  last  time  period  is  calculated  as 


Vp,T,a  ~  YicleA  xp,T,a,a{cpj,a  I p,T,a,a )■ 

The  value  of  the  option  for  time  periods  other  than  the  last  one  is 


Vp,t,a  YiaeAxp,t,a,a(Cp,t,a  Ip,t,a,a)  T 


р, t+i,a 

с, rAt  5 


(2) 

(3) 


where  the  denominator  erAt  discounts  future  values  using  the  riskless  rate  of  return 
r.  In  addition  to  the  valuation,  the  model  constrains  the  total  investment  in  any  period  to  be 
less  than  the  total  available  budget  and  enforces  consistency  such  that  switching  in  one  time 
period  is  consistent  with  the  state  in  the  next  time  period. 


The  objective  is  to  maximize  the  total  expected  value  of  the  portfolio.  The  objective 
function  is 


maxZ  =  Y,psp SdE a  Vp,i,ixp,i,i,d-  (4) 

The  mathematical  model  presented  is  a  discrete  stochastic  program  that  can  be 
solved  for  the  optimal  decisions  for  the  single  scenario  represented  by  the  random  variable 
9pt.  The  value  of  0p  t  is  generated  by  the  Monte  Carlo  simulation  method  and  using  the 
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method  of  Iman  and  Conover  (1982)  so  that  the  resulting  random  variables  are  correlated 
according  to  the  correlation  matrix  pxy. 

The  algorithm  on  the  right-hand  side  of  Figure  1  has  two  loops.  The  first  loop 
conducts  N  Monte  Carlo  simulations.  The  results  from  the  N  simulations  are  aggregated, 
and  then  a  decision  heuristic  is  applied  to  select  which  project  options  to  invest  in  for  time 
period  t.  The  budget  and  project  option  variables  xPita&  are  updated.  The  second  loop  is 
followed  if  there  is  budget  left  in  time  period  t  such  that  it  may  be  worthwhile  to  make 
another  option  decision.  If  not,  then  the  time  period  is  advanced  and  N  Monte  Carlo 
simulations  are  performed  for  the  next  time  period.  Thus,  if  we  conduct  N  =  100  simulations, 
we  do  this  for  each  time  period  such  that  with  a  planning  horizon  of  four  years  (i.e.,  T  =  4), 
then  we  perform  a  total  of  400  simulations. 

Each  Monte  Carlo  simulation  provides  the  optimal  portfolio  for  the  randomly 
generated  scenario,  so  the  method  requires  performing  many  simulations  to  generate  a 
profile  for  decisions  in  the  planning  horizon.  The  decision  of  which  options  to  exercise  is 
determined  by  a  heuristic.  The  heuristic  is  a  simple  best-first  heuristic.  The  heuristic  solution 
approach  does  not  guarantee  an  optimal  portfolio  decision  policy.  We  do  not  explore  in  this 
paper  the  performance  of  one  heuristic  over  another;  however,  future  research  may  be 
conducted  to  determine  which  heuristic  performs  better  and  under  what  circumstances. 

Illustrative  Example 

We  present  a  case  study  to  illustrate  the  enterprise  transformation  framework.  ABC 
company  is  a  small-  and  medium-sized  engineering  and  manufacturing  firm  of  defense 
systems.  It  is  a  low-volume  and  high-mix  manufacturer,  performing  the  design,  development, 
and  system  integration  for  electromechanical  systems  in  the  defense  industry.  The  company 
also  has  a  growing  capability  for  research  and  development  (R&D),  including  R&D  under 
contract  to  larger  companies.  The  company’s  revenues  stand  at  approximately  $15  million 
annually,  which  the  company  expects  to  grow  to  $50  million  over  the  next  five  years  through 
a  strategy  of  further  developing  and  exploiting  its  R&D  capabilities.  The  company  envisions 
an  enterprise  that  has  a  more  visible  role  in  the  early  phases  of  defense  system 
development  and,  consequently,  a  larger  part  of  the  value  chain.  Management  has  identified 
a  strategy  for  achieving  this  growth  and  is  concerned  about  obstacles  that  may  prevent  the 
fundamental  changes  to  transform  the  company.  The  strategy  includes  achieving  greater 
efficiency  of  operations,  better  integration  of  internal  systems  so  they  can  have  better 
coordination  of  activities,  and  better  integration  with  customers  in  order  to  work  in  a  more 
open,  collaborative  environment.  A  company  of  about  100  employees  is  different  from  a 
company  of  250.  The  projects  identified  involve  IT  infrastructure  investment,  enterprise 
systems  investment,  reorganization  investment,  and  training  investment.  The  data  required 
to  create  a  real  options  portfolio  is  presented  in  Table  2.  For  each  project,  three  operating 
modes  are  defined.  The  investment  to  switch  from  one  operating  mode  to  another  is 
provided  as  well.  The  standard  deviation  of  the  random  variable  is  required  for  the  Monte 
Carlo  simulation.  The  project  correlations  are  shown  in  Table  3. 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-244- 


Table  2.  Required  Input  Data  for  Portfolio  Selection  (Notional  Values) 


Ipaa' 

Random  Variable 

al 

a2 

a3 

ep,i,i 

F 

VC 

ap 

PI 

al 

Maintain  current  organization  of  functional 
departments  (i.e.,  delay) 

0 

0.8 

1.2 

4 

0 

0.9 

0.7 

a2 

Reorganize  into  program  groups 

0.2 

0 

0.2 

4 

0.2 

1 

0.7 

a3 

Expand  reorganization  to  marketing  and  other 
departments 

0.3 

0.1 

0 

4 

0.3 

1.2 

0.7 

P2 

al 

delay 

0 

0.6 

0.7 

0.35 

0 

0 

0.15 

a2 

COTS  -  local 

0 

0 

0.1 

0.35 

0 

1 

0.15 

a3 

Option  for  HR,  CRM,  and  ERP  (growth) 

0 

0 

0 

0.35 

0 

1.1 

0.15 

P3 

al 

delay 

0 

0.45 

0.55 

0.3 

0 

0 

0.3 

a2 

SolidWorks  upgrade  to  SW  Simulation  Premium  for 

0.05 

0 

0.08 

0.3 

0 

1 

0.3 

a3 

Pro-E 

0.05 

0.05 

0 

0.3 

0 

1.1 

0.3 

P4 

al 

delay 

0 

0.76 

1 

0.6 

0 

0 

0.25 

a2 

ADP-EZ  Payroll  (Payroll) 

0.1 

0 

0.15 

0.6 

0 

1 

0.25 

a3 

ADP  -  EZ  Labor  (labor  hr  tracking) 

0.1 

0.1 

0 

0.6 

0 

1.2 

0.25 

P5 

al 

delay 

0 

0.2 

0.45 

0.1 

0 

0 

0.08 

a2 

DOORS  or  Requirement  Mgmt  Tool 

0 

0 

0.22 

0.1 

0 

0.8 

0.08 

a3 

CORE  Requirement  Mgmt  Tool 

0 

0 

0 

0.1 

0 

0.9 

0.08 

Table  3.  Correlation  Matrix 


PI 

P2 

P3 

P4 

P5 

PI 

1 

0 

0.25 

-0.25 

0.25 

P2 

1 

0.5 

-0.25 

0.5 

P3 

1 

0.5 

0.5 

P4 

1 

0.5 

P5 

1 

The  model  was  executed  with  N  =  100  Monte  Carlo  simulations  for  each  decision 
epoch.  Figure  2  shows  the  results  of  the  first  100  simulations  in  Period  1.  Using  a  best-first 
heuristic,  the  decision  is  made  to  do  the  switch  defined  by  xllx3  because  Project  1 
switching  from  operating  mode  a?  (the  initial  operating  mode)  to  a3  has  the  highest 
percentage.  Figure  2  indicates  that  making  this  switch  was  part  of  the  optimal  solution  in 
75%  of  the  simulated  scenarios.  Notice  in  this  example  that  x3X11  has  an  equal  percentage. 
The  tiebreaker  was  decided  because  switching  in  Project  1  was  a  positive  switch  rather  than 
the  “do  nothing”  option  represented  by  Project  3. 


ru.i  m'--ii,.riE:w'rY]iU| 
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Period  1 


■  Project  1 

■  Project  2 

■  Project  3 

■  Project  4 

■  Projects 


Figure  2.  Aggregation  of  First  N  Monte  Carlo  Simulations 


Following  the  first  decision  epoch,  the  budget  is  updated  and  the  decision  variable 
xi,i,i, 3  is  set  to  be  1,  meaning  that  this  switch  has  been  decided  upon  and  fixed.  The  next 
iteration  of  100  Monte  Carlo  simulations  is  conducted  and  the  results  aggregated  to  reveal 
Figure  3.  Project  1  switching  to  Option  3  is  now  higher,  but  in  a  small  percentage  of 
scenarios,  the  optimal  decision  was  to  contract  the  project  by  switching  to  operating  mode 
a2.  The  next  decision  is  to  continue  delaying  Project  3  and  to  do  a  pilot  study  for  Project  5. 


12  3 

a' 


■  Project  1 

■  Project  2 

■  Project  3 

■  Project  4 

■  Project  5 


Figure  3.  Aggregation  of  First  N  Monte  Carlo  Simulations 
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Discussion  of  Results 

The  discrete  stochastic  model  for  creating  a  portfolio  is  intractable  due  to  the  curse  of 
dimensionality,  so  we  resorted  to  a  heuristic  algorithm  utilizing  a  Monte  Carlo  simulation.  In 
the  heuristic  algorithm,  we  ran  a  set  of  simulations,  aggregated  the  results,  and  applied  a 
heuristic  to  make  a  decision.  We  implemented  a  rather  unsophisticated  best-first  algorithm. 
We  did  not  investigate  other  heuristics  or  the  performance  of  this  heuristic;  we  leave  this  for 
future  research.  In  applying  the  heuristic,  it  is  noted  that  in  no  cases  was  a  switch  optimal  in 
all  simulated  scenarios.  An  interesting  research  question,  which  we  have  not  yet 
investigated,  is  whether  we  can  analyze  the  simulation  runs  and  gain  insight  into  scenarios 
in  which  one  switch  is  better  than  another. 

Conclusions 

The  paper  was  motivated  by  providing  acquisition  decision-makers  with  tools 
necessary  to  make  informed  decisions  in  selecting  a  portfolio  of  enterprise  projects.  The 
paper  described  a  method  to  recast  enterprise  projects  in  terms  of  options  on  the  project  to 
make  explicit  the  decision  flexibility  available  to  the  decision-makers.  The  main  contribution 
is  a  real  options  valuation  model  that  is  applied  to  the  portfolio  as  a  whole,  rather  than  each 
project  individually.  The  complexity  of  selecting  a  good  portfolio  is  performed  with  an 
algorithm  and  mathematical  model.  Valuation  of  the  portfolio  of  options  is  performed  with  a 
Monte  Carlo  simulation.  The  approach  was  illustrated  with  a  case  study  of  a  small 
manufacturer.  Future  work  will  investigate  applicability  of  the  model  to  the  warfighter  side  of 
the  DoD  that  is  non-financial  and  driven  by  acquisition  of  capabilities. 

References 

Bardhan,  I.,  Bagchi,  S.,  &  Soustad,  R.  (2004).  Prioritizing  a  portfolio  of  information  technology 
investment  projects.  Journal  of  Management  Information  Systems,  21(2),  33-60. 

Brosch,  R.  (2008).  Portfolio  of  real  options.  Berlin,  Germany:  Springer  Verlag. 

Dickinson,  M.  W.,  Thornton,  A.  C.,  &  Graves,  S.  (2001).  Technology  portfolio  management: 

Optimizing  interdependent  projects  over  multiple  time  periods.  IEEE  Transactions  on 
Engineering  Management,  48(4). 

Engle,  A.,  &  Browning,  T.  (2006).  Designing  systems  for  adaptability  by  means  of  architecture 
options.  In  16th  Annual  International  INCOSE  Symposium  Proceedings. 

Giachetti,  R.  E.  (2010).  Design  of  enterprise  systems:  Theory,  architecture,  and  methods.  Boca 
Raton,  FL:  CRC  Press. 

Iman,  R.L.,  &  Conover,  W.  J.  (1982).  A  distribution-free  approach  to  inducing  rank  correlation  among 
input  variables.  Communications  of  Statistica-Simula  Computa,  11,  311-334. 

Koenig,  P.  (2009).  Real  options  in  ship  and  force  structure  analysis:  A  research  agenda.  American 
Society  of  Naval  Engineers,  4,  95-106. 

Kulaitilaka,  N.  (1988).  Valuing  the  flexibility  of  flexible  manufacturing  systems.  IEEE  Transactions  on 
Engineering  Management,  35(4),  250-257. 

Trigeorgis,  L.  (1996).  Real  options:  Managerial  flexibility  and  strategy  in  resource  allocation. 
Cambridge,  MA:  MIT  Press. 

Uchytil,  J.,  Housel,  T.,  Horn,  S.,  Mun,  J.,  &  Tarantino,  E.  (2007).  AEGIS  and  ship  self-defense  system 
(SSDS)  platforms:  Using  KVA  analysis,  risk  simulation  and  strategic  real  options  to  assess 
operational  effectiveness.  In  Proceedings  of  the  Fourth  Annual  Acquisition  Research  Program. 
Monterey,  CA:  Naval  Postgraduate  School. 

Wang,  T.,  &  de  Neufville,  R.  (2006).  Identification  of  real  options  in  projects.  Presented  at  the  4th 
Conference  on  Systems  Engineering  Research,  Hoboken,  NJ. 


ACQUISITION  RESEARCH  PROGRAM: 
CREATING  SYNERGY  FOR  INFORMED  CHANGE 


-247- 


ACQUISITION  RESEARCH  PROGRAM 

GRADUATE  SCHOOL  OF  BUSINESS  &  PUBLIC  POLICY 

NAVAL  POSTGRADUATE  SCHOOL 

555  DYER  ROAD,  INGERSOLL  HALL 

MONTEREY  CA  93943 

www.acquisitionresearch.net 


Acquiring  Enterprise  Systems  as  a 

Portfolio  of  Real  Options 


Ronald  E.  Giachetti,  Ph.D. 

Professor 
Systems  Engineering 
Naval  Postgraduate  School 
Monterey,  CA 
regiache@nps.edu 


1 


Motivation 


■  Enterprise  architecture  (EA)  is  designed  and  acquired  in 
an  evolutionary  manner  over  a  long  period  of  time 

♦  Complex  systems 

♦  High  uncertainty  and  project  risk 

♦  Changing  needs,  technology,  resources  (budgets),  and  priorities 

■  Researching  Two  Approaches: 

1.  analyze  architectures  before  constructing 
them  --  reduce  uncertainty 

2.  Structure  system  architecture  to  support 
flexible  acquisition  coupled  with  decision 
tools  fo  react  to  new  information  as 
project  unfolds 
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Enterprise  Systems  in  a  Large  Beverage  Co. 

Do  you  think  it  was  architected  like  this? 


Pillar  One  Detail  QTG  Legacy  and  One  Up  Systems  Flow 

Layer:  Pillar  1 


•  Bottlers 


Master 


Tr^ 

Promotio 


WklyOp 


IPLDO 


svoc  - 

QyPndcl 

Cod® 


SAPSD 


PTMS 


ToiFlom:  MSAFA 
FA 


(inv/Ordertf 


|  SAP  / 
f  Financial^  / 


M204 

Customer  Order 


Broker 


Vertex 


CDVAL 


FLUS 

Legacy 

PCNA 

legacy 

QFB 

Legacy 

Tropicsna 

One  Up 

* 

Under  Discussion 
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Today's  Shipboard  Environment 

( D  i  rect  i  nte  liace  s .  unique  sol  Lrtions .  wea  k  cross  -dona  i  n  i  ntegration) 
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Source:  “Surface  Navy  Combat  Systems  Engineering  Strategy”  Briefing  to  NPS.  Kathy  Emery,  Chief 
Architect,  PEO  Integrated  Warfare  Systems,  4  March  2010 
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Architecture  &  Engineering 


Architecture  defines  the 
parameters  and  overall  constraints 

•  Holistic 

•  Satisficing 

•  Heuristic 

•  Ambiguous 
'fuzzy'  needs 

•  High 
uncertainty 


Engineering  optimizes 
the  parameters  subject  to 
the  constraints 

•  Reductionist 

•  Optimizing 

•  Algorithms 

•  Requirements 

•  Less  uncertainty 
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Enterprise  Systems  Engineering  Process 


JNPSJI 

fwflSttNTlA  PER  SCIINTIAM 
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understand 


design 


understand 


design 


understand 


design 


build 


test 


understand 


design 


build 


test 


understand 


design 


build 


test 


understand 


design 


build 


test 


understand 


design 


build 


test 


Kick-Off  Meeting  PAcceptec!n  Analysis  Review  Alternative  Design  Review 


Go  Live 

▲ 


Project  Initiation  I 


Project  Planning 


Analysis 


Generate  & 
Evaluate 
Alternatives 


Design 


Construction 


Project  Charter 
Business  Case 


] 


Project  Plan 
Project  Schedule 
Budget  Plan 
QA  Plan 
Risk  Mgmt  Plan 
Configuration 
Control  Plan 


] 


Candidate  Analysis  Desjgn  Mode,s  Systems 

Change  Management 
Plan 
Test  Results 
Training  materials 
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EA  Design  Evolution 

■  EA  does  not  start  with  a  blank  sheet  of  paper 


it  is  the  transformation  of  existing 
architecture  (As-ls)  to  a  target 
architecture  (To-Be) 

In  large  organizations,  it  is 
hierarchical,  involving  many  levels 
of  planning  and  system  design 


Strategic 
Plan 


BE  A 
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Enterprise  Architecture 


DoD  IT 
Strategic 
Plan 


Enable  net-centric  operations 
•  Information  as  strategic  asset 

•  Interoperable  infrastructure 
•  Assured  information  access 
•  Return  on  investment 


Business 

Enterprise 

Enterprise 

Transformation 

Architecture 

Plan 

(BEA)  J 

(ETP) 

DON 
Strategic 
Plan 


alignment 


DON 

Enterprise 

Architecture 


Naval 

Transformation 

Roadmap 


{ ...  Army } 

{ ...  Air  Force  } 


NAVAIR 


X 


SPAWAR 
(IT  Dominance) 


Navy  SYSCONs 
Have  architectures 
&  plans 
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Tech  Authority  Approach  for  Controlling  Navy  IT 


Information  Dominance 
Mission  Needs  Document 


~\ 


N2N6 


Information  Dominance  SoS 
Architecture  &  Requirements 

Portfolio  Requirements 
Portfolio  l/F  Control  Doc 
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SPAWAR 

CHENG 


NGEN  Services  (38) 


NGEN  Systems  (24) 
NGEN  Cl  ("box"  design) 


PEO/PMW 

Assured  traceability  from  mission 
needs  to  implementation 


Flexible  Realization  of  EA 


Define  projects  as  portfolio  of  real 
options 

♦  Modularization  of  EA  into  projects  - 
breadth  and  time  dimensions 

♦  Active  management  of  systems 
engineering  project  via  real  options 


■  Incorporate  alternatives  into 
planning  process 

■  Defer  some  decisions  until 
uncertainty  can  be  resolved 
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Phased  modular  deployment 


PMESTANT1A  PER  self  NTIAM 


GCSS-MC  Enterprise 


* 


-TERRESTRIAL  LINK 


CONUS 


PISN  POP 

(S4PR1 

NIPfthuTi 


STEP 

TELEPORT 


Enterprise  Release  1.1 

HI5A  Production 
DISA  Non-Production 
Carps  hia  Non -Production 
Mobi  a  Train  inq  Suites 

Marine  Corps  Logistics 
Modernization  is  the  backbone  for  all 
logistics  information  required  by  Marine 
Air  ground  Task  Forces  (MAGTFs). 
Oracle  11  i  business  suite. 


Deployed  Release  1.2 

DISA  Development  for  Re  ease '  _ 
MEF 
UEU 

Disaster  Recover, 


GCSS-MC  BLOCK  I 
iARRISON/TACTICAL  EMPLOYMENT 
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Converting  Project  to  RO 


Global  Combat  Support  System  Marine  Corps  (GCSS-MC)  - 
Technology  enabler  for  Logistics  Modernization  strategy;  An  Oracle  1 1  i  e- 
business  suite  to  replace  multiple  legacy  systems. 


GCSS-MC 


Delay 


Asset  Tracking 
Maintenance  Mgmt  Sys 
Storage,  Retrieval  Tracking  Sys 


Asset  Tracking 
Maintenance  Mgmt  Sys 
Storage,  Retrieval  Tracking  Sys 


Delay 

Upgrade  to  Oracle  12 

Integrate  Auto  ID 
Technology 
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Binomial  Tree 


t  =  1 


t  =  2 


t  =  3 


t  =  4 


0 


P.4,1 


0. 


P.4.2 


e. 


P.4,3 


0. 


P.4,4 


e. 


P.4,5 


6. 


P,4,6 


0. 


P.4,7 


0. 


P.4.8 


A  single  project  p 
whose  associated 
random  variable  has 
two  possible 
outcomes'. 

Improvement  (UP)  or 
Worse  (DOWN) 

Requires  15 
calculations  for 
option  value  by  time 
period  and  scenario 
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Problem  Tractability 


In  2011,  Navy  has  36 
active  enterprise  system 
projects 


Interrim, 
4 


In  a  portfolio  of  36  projects  and  a  4- 
y ear  planning  horizon,  there  are  more 
than  68  billion  possible  states  after 
the  first  year 


Provides  a 
capability(ies) 
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Model 


PRMSTANT1APERSCI  intiam 


Decision  Variable: 


Net  present  value  of  costs  and  benefits  for 
project  p  in  time  period  t 

i  denotes  state  of  project  (e.g.,  /'  =  1  for 
delay/abandon;  /=  2  for  pilot;  /=  3  for 
expand) 


Denotes  the  decision  for  project  p  and  time  period  f, 
whether  to  switch  from  state  a  to  state  a’ 


X 


ptaa' 


1  denotes  switch  from  a  to  a' 
0  otherwise 
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Converting  Project  to  RO 


Global  Combat  Support  System  Marine  Corps  (GCSS-MC)  - 
Technology  enabler  for  Logistics  Modernization  strategy;  An  Oracle  1 1  i  e- 
business  suite  to  replace  multiple  legacy  systems. 


xpin 


Xp211 


ai 

Delay 


a2 

Asset  Tracking 
Maintenance  Mgmt  Sys 
Storage,  Retrieval  Tracking  Sys 


a2  Xp212 
Asset  Tracking 
Maintenance  Mgmt  Sys 
Storage,  Retrieval  Tracking  Sys 


Delay  ai 

Upgrade  to  Oracle  12 

Integrate  Auto  ID 
Technology 
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Correlation  -  Portfolio  Effects 


PRMSTANT1APERSCI  intiam 


In  a  portfolio  of  projects,  it  is  likely  that  some  project  benefits/costs 
are  correlated 


Estimate 

correlation 


Conelation 

Value 

Correlated 

i  (-i) 

High 

0.75  (-0.75) 

Moderate 

0.5  (-0.5) 

Minor 

0.25  (-0.25) 
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0 
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P5 
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1 
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0.25 
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P2 

1 

0.5 
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P3 

1 

0.5 
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P4 

1 

0.5 

P5 

1 
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max  Z  =  ^  Y 

(i) 

Discrete  Stochastic 

=  /pwa  +  V"  V  v 6  Pt€T,»€  A 

(2) 

Optimization  Model 

1  jj.r.a,  “  ^  J’jj.r.fl.Hi'  (rp.T.or  - 

0) 

vpt  Pj  =  T,ti£  >1 

=  ^  (^t,(pOr  —  IpAjfu i?r  ) 

flipJ 

(4) 

Equations  to 
value  the 

,  OWW 
*** 

Vp  tP,t=  1..T-  l,a  e  .4 

options  and 
enforce  the 

r  ■**«.»."'  =  1  v  p  e  P,l  €  T 

XiMjj,-  =0  v,.6  P,n  ^  iV  e  4 

(fi) 

(6) 

budget 

Y  XP*A°  =  51  vj» €  p,  ( e  (2,T] , a  e  >i 

rtr£.4 

(7) 

constraint 

BptM=fa-  ^  (2p,M.<r  ~  Jpjtl.a') 

P*P^'€A 

(8) 

B%  —  iff— I  "t  A  —  ^  .a.a'  V  f  £  ,2..  /  I 

fiEP.a.a'Evi 

(fl) 

fl*  >  0  V  t  G  r 

(10) 

ifii^  e  [0, 1]  vp  e  P, i  e  r,  a  e  A.ar  e  .4 

(11) 

,o  ,o.T  Ss  l 

U2) 

vi  e  7\  «  e  A 


Ronald  E.  Giachetti 
May  18,  2012 


Slide  18 


Monte  Carlo  Simulation 


PRMSTANT1APERSCI  intiam 


Monte  Carlo 
•  Simulations 


Algorithm  generates  random  scenario 
and  find  optimal  solution  for  that 
scenario 

Do  this  for  many  scenarios  - 
aggregate  into  a  distribution 


Given  distribution,  use  a 
heuristic  to  select  "best" 
option 
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Enterprise  System 
Planning  Algorithm 


Projects  and  architecture 
are  input 


Define  real  options  on 
each  project 


Cost/benefit  analysis  for 
each  operating  mode 


Projects  correlation  in 
portfolio 


Cluster  projects  via  goals 


Define  constraints 
(budaet/vear.  resources) 


Do  N  Monte  Carlo  simulations 
using  discrete  stochastic  options 
optimization  model  -  aggregate 
^results  and  apply  heuristic _ 
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Enterprise 

List  of  Potential 

Architecture 

Projects 

_ J  Portfolio  of  projects 

planned  to  reach 
target  enterprise  state 


Set  number  of  simulations 
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Input  Data 


PRMSTANT1APERSCI  intiam 


Small  Defense  Contractor  -  low-volume,  high-mix 
manufacturer  of  electro-mechanical  systems 
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0.05 

0.05 

0 

0.3 

0 
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Results  -  Best  First  Heuristic 


PRMSTANT1APERSCI  intiam 


Period  1 


100% 

90% 
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■  Project  1 

■  Project  2 

■  Project  3 

■  Project  4 

■  Project  5 


y  / 

ptaa 

*1113  =  1 


Project  #1,  in  time  period  #1 
Switch  to  operating  mode  3 


Period  2 


I  Project  1 
I  Project  2 
i  Project  3 
I  Project  4 
Project  5 


*5212  =  1 
*4212  =  1 


Project2  #4  and  #5,  in  time  period  #2 
Switch  to  operating  mode  2 
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Time  Horizon  Results 


PRMSTANT1APERSCI  intiam 


Expected  Value,  assuming  optimal  decisions 


$100  $108  $116  $124  $132  $140  $148  $156  $164  $172  $180 

Expected  Value 


The  entire 
distribution  of 
possible  outcomes 
influences 
decisions,  not  just 
the  expected 
value 

Risk -informed 
decision  making 


Each  year,  as  uncertainty  is  resolved  (what  happened  during 
the  past  year?)  -  redo  plan  going  forward  and  revise  plans 
according  to  new  information. 
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Future  Work 


PRMSTANT1APERSCI  intiam 


■  Applicability  to  the  warfighter  side  of  DoD  that  is  non- 
financial  and  driven  more  by  acquisition  of  capabilities 
via  evolutionary  acquisition  process 

■  Model  is  intractable  for  realistic  size  problems,  opportunity 
for  improvement  in  algorithm  over  the  heuristic  approach, 
or  experiment  with  different  heuristics 

■  Opportunities  to  investigate  real  options  designed  IN  the 
architecture  versus  real  options  ON  the  project 
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Conclusions 


Motivation:  Flexibility  is  being  left  on  the  table  -  rethink  projects  and 
system  architectures  in  terms  of  “options”  can  help  recapture  and  use 
this  flexibility 

♦  Decision  makers  do  think  about  these  types  of  options,  but  the  informal  approach 
may  miss  options,  is  not  based  on  valuation,  and  human  cognitive  limits  in 
evaluating  multiple  projects/options  concurrently 

■  Broad-based  Method 

♦  Need  flexible  methods 

♦  Need  means  to  predict  enterprise  architecture  performance 

♦  Need  means  to  plan  EA  evolution  as  series  of  projects  /  real  options 

■  Model  goes  hand-in-hand  with  evolutionary  acquisition  of  capabilities 

■  Design  EA  in  terms  of  modules  to  implement  real  options  in  planning 
Consider  a  project  portfolio  because  decisions  are  inter-related 
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